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About three-quarters of the fastest-growing jobs in the next decade will require some 
postsecondary education, according to Bureau of Labor Statistics projections. In 2015, adults 
ages 25-64 without postsecondary credentials were already disadvantaged, experiencing twice 
the unemployment rate of those with a high school credential or greater. In addition, many 
high school graduates are underprepared for postsecondary education and are placed in 
developmental education classes. By one estimate, community colleges referred approximately 
three-fifths of first-time enrolling students to at least one developmental math class and one- 
third to at least one developmental reading class (Bailey, Jeong, and Cho 2010). 


In recent years, two-year colleges have tried various strategies to improve student outcomes 
and remediate skill deficits in order to promote academic and employment success. Promising 
approaches include career pathways, dual-enrollment, competency-based credentialing, 
sectoral strategies, wrap-around services, and acceleration. Taken together, the papers on this 
panel provide important insights into these innovations in higher education. The proposed 
discussant lends a federal perspective that will elevate the conversation. 


One of the more influential studies in this field was the research on Washington State’s 
Integrated Basic Education and Skills Training (I-BEST) program, which provided promising 
evidence on the effectiveness of enhanced career pathways programs (Wachen, Jenkins, and 
Van Noy 2010; Zeidenberg, Cho, and Jenkins 2010). The I-BEST findings inspired various 
replication efforts. Two of the papers in this panel report impact findings from I-BEST-like 
programs, utilizing different methods and in different state contexts. The first paper presents 
the results of an experimental study at 10 colleges in Maryland, Connecticut, Texas, and 
Georgia. The second paper shares results of a quasi-experimental analysis on 54 colleges in 
Illinois, Kansas, Kentucky, and Louisiana. Both of these studies are supported by robust 
implementation research, which will give critical context to the impact findings. The third paper 
presents impact findings from another state-grown career pathway effort that incorporates 
sectoral strategies; the analysis benefits from eight years of administrative data for 22 colleges 
in Arkansas. The final paper examines the results from a three-college consortium’s effort to 
implement competency-based approaches in the growing information technology sector using 
quasi-experimental approaches and dosage analysis. 


The proposed discussant will lend a federal policymaking perspective and be able to connect 
these efforts with the federal government’s goals for adult learners. Her background in 
research and evaluation will allow her to provide a knowledgeable assessment of the quality of 
the research and her position within the policymaking process will allow her to judge the 
implications of the findings. 


These evaluations are supported by various public and private funding streams. Federal support 
comes from the Department of Labor’s Trade Adjustment Assistance Community College and 
Career Training (TAACCCT) and Workforce Innovation Fund (WIF) grants. Private support is from 
the Winthrop Rockefeller, Ford, Annie E. Casey, Gates, Joyce, Open Society, Kresge, Arthur 
Blank, Woodruff, University of Phoenix, and Kellogg Foundations. This range of support 
demonstrates that there is broad interest in understanding promising approaches to support 
student success in two-year colleges. 


Symposium Organizer: Theresa Anderson (tanderson@urban.org), Research Associate, the 
Urban Institute 


Symposium Discussant: Cheryl Keenan (Cheryl.Keenan@ed.gov), Director of the Adult 
Education and Literacy Division, Office of Career, Technical, and Adult Education, U.S. 
Department of Education 


Note: The presenting author is underlined in each abstract. 


Paper 1 Title: Accelerating Connections to Employment: A Collaborative Approach to Career 
Pathways Training and Workforce Development 


Authors: Yvette Lamb (Yvette.Lamb@icfi.com) and Christina McHugh 
(Christina.McHugh@icfi.com) — ICF International 


Abstract (484 words): 


The Accelerating Connections to Employment (ACE) National Evaluation study seeks to 
determine and improve the effectiveness of workforce training programs targeting low-skilled, 
low income individuals in the workforce system. ICF International is implementing a mixed 
method evaluation of the ACE program, including a randomized control trial designed to assess 
key outcomes including employment, wages, and job retention. Funded by the Department of 
Labor’s Workforce Innovation Fund, the ACE study exemplifies the Federal government’s focus 
on evidenced-based policy. Building upon Washington State’s Integrated Basic Education and 
Skills Training (I-BEST) model, the ACE program has attracted the attention of both state and 
federal policymakers interested in assessing its costs and scalability nationwide. 


ACE, led by the Baltimore County’s Department of Economic and Workforce Development, aims 
to reduce poverty by linking education, training, and workforce services to create skill-building 
opportunities and career pathways for low-skilled individuals. The ACE model seeks to foster 
systems integration and increased coordination by bringing together ten community colleges, 
nine Workforce Boards, and employer partners across four states — Maryland, Connecticut, 
Texas, and Georgia. Each partner has a critical role to play in the provision of services. 
Workforce Boards work with the colleges to streamline and better coordinate job placement, 
supportive services, and funding. Community colleges deliver ACE training and collaborate on 
assessment, coaching, and job placement activities. Following the I-BEST model, ACE training 
included occupational instruction in high-demand fields chosen based on labor market 
information, along with a basic skills instructor to address basic skills challenges. Employers 
provide information on labor market demands and skills needs and offer internships and job 
opportunities. 


Key research questions for the ACE evaluation include: 1) do ACE participants realize higher 
rates of employment, higher wages, and increased job retention relative to individuals receiving 
business-as-usual services; 2) how did sites implement the components of ACE and how close 
was this implementation to the program model as envisioned; 3) which components worked as 
intended and which innovations to the original program model proved necessary. To answer 
the first question, ICF conducted a randomized controlled trial including approximately 2,560 
individuals randomized into either a treatment group, receiving the ACE program, or a control 
group, receiving business-as-usual services. To provide context and identify best practices in 
program implementation, ICF also conducted a series of site visits, staff interviews, student 
focus groups, and case studies. 


During this symposium presentation, the ACE evaluation team will detail the study design and 
describe the key elements of the ACE model. We will present the findings of the randomized 
controlled trial component of the evaluation regarding the program’s employment and wage 


outcomes for low-income individuals. Results will be based on preliminary analyses of self- 
reported and unemployment insurance data, with the final report scheduled for mid-2017 
release. In addition, presenters will triangulate outcomes data with qualitative findings to 


provide context and suggest best practices and lessons learned for how, when, and for whom 
the ACE model works. 


Paper 2 Title: Final Impact Findings from Accelerating Opportunity 


Authors: Theresa Anderson (Urban Institute, tanderson@urban.org), Lauren Eyster (Urban 
Institute, leyster@urban.org), Daniel Kuehn (Urban Institute, dkuehn@urban.org), Burt Barnow 
(George Washington University, barnow@gwu.edu), and Robert |. Lerman (Urban Institute, 
blerman@urban.org). 


Abstract (994 words): 


Background: The Accelerating Opportunity (AO) initiative aimed to provide adults with low 
basic skills a pathway toward meaningful and family-supporting careers. Typically, adults with 
low skills are ill-prepared for educational and employment success. Nationally, 11 percent of 
adults lack a high school diploma or General Educational Development (GED) credential. The 
average unemployment rate in 2015 for adults ages 25-64 without high school credentials was 
78 percent higher than the unemployment rate than those with some college or greater. 


Adults without high school credentials may continue their education by enrolling in adult 
education programs. However, these programs rarely help adults progress into postsecondary 
education or advanced training. Instead, adult education programs—operated by community 
and technical colleges, school districts, and community-based organizations—are often 
oriented toward helping students obtain a GED or adult high school diploma or toward 
improving English language skills. They typically have few links to postsecondary education or 
advanced training that yield recognized occupational credentials that would help students get 
well-paying jobs. 


Adults without high school credentials are not the only population that struggles with low skills. 
Many high school graduates are underprepared for postsecondary education and are placed in 
developmental education classes, which can take multiple semesters and cause students to use 
up their federal financial aid allocation before they earn any college credit. Various states have 
undertaken efforts to reform developmental education in recent years to improve student 
retention and success. 


Intervention: AO provides one potential solution to the challenges facing adults who would 
otherwise be in adult education or developmental education programs. AO is a scale-up effort 
for the Integrated Basic Education and Skills Training (I-BEST) model, developed in Washington 
State. It was designed to help low-skilled adults, particularly those without high school 
credentials, succeed in career and technical education at two-year colleges. In the AO initiative, 
students with low basic skills or who lack a high school credential co-enroll in adult education 
and technical courses and experience team teaching, where basic skills and technical instructors 
jointly provide content in the classroom. Students’ course-taking is organized into career 
pathways in high-growth, high-wage industries, as informed by local labor market information. 
These 12-credit pathways must lead to stackable, marketable, industry-recognized credentials. 
Students also receive additional support services, often the attention of an AO coach or 
navigator and other forms of barrier remediation. 


Research Question: This paper shares the results of a study of the impacts of AO on students’ 
education and employment outcomes, which seeks to answer the following research questions: 


¢ Who were the AO students and how did they differ across states? 
® How did colleges’ AO pathways and recruitment efforts shape who enrolled in AO? 
® How did characteristics of AO students change over time? 


© Were AO students more likely to earn credentials and credits than similar individuals 
who did not enroll in AO? 


¢ Were some groups of AO students more successful than others in their educational 
outcomes? 


© Were AO students more likely to have positive employment and earnings outcomes 
than similar individuals who did not enroll in AO? 


¢ Were some groups of AO students more successful than others in their labor market 
outcomes? 


Setting: This study took place in 54 two-year community and technical colleges across four 
states: Illinois, Kansas, Kentucky, and Louisiana. Only states in which the postsecondary system 
governs adult education were eligible to participate in AO. 


Participants: AO enrollees were students with low basic skill levels, meaning that they scored 
on basic skills test within National Reporting System levels 4-6 (6th- to 12th-grade equivalency 
levels) on math, reading, or writing or National Reporting System levels 5-6 in English-language 
skills (high-intermediate to advanced levels). Though the initiative originally targeted adults 
without high school credentials, the only formal eligibility criterion was that students test 
within the qualifying skill levels. Therefore, eligible students may or may not have had 
secondary school credential at program entry. Illinois and Louisiana primarily recruited adults 
without high school credentials, while Kansas and Kentucky recruited more heavily from 
developmental education and low-skilled career and technical education students who 
generally had completed secondary school. 


Research Design, Data Collection, and Analysis: The evaluation, led by the Urban Institute and 
the George Washington University, includes an implementation study, a quasi-experimental 
impact analysis, and a cost-benefit analysis. This paper focuses on the results of the impact 
study. 


For the impact study, the research team utilized linked state education and earnings records to 
track AO and comparison group students before, during, and after college enrollment. Through 
propensity score matching, AO students were matched with students enrolled in the same 
occupational area and with similar characteristics (demographic, educational background, 
baseline test scores, cohort, and earnings and employment histories) who did not have the 


opportunity to take part in the AO program. Estimates are shared by state and by recruitment 
source (i.e., adult education, developmental education, or career and technical education). 


The outcomes of interest in is paper are credits earned, academic credentials earned, 
persistence in postsecondary education, time to completion of meaningful educational 
milestones, employment, and earnings. The final impact report will be released in late 2016. 


Findings: Interim impact results on short-term educational outcomes, completed in 2015, 
showed that AO students earned more credentials than matched comparison group members 
earned, but that the impacts on the number of credits earned varied by state and recruitment 
source. This analysis of longer-term education and labor market outcomes will provide a more 
complete picture about whether or not AO students were able to “get ahead” because of the 
program. The research team will contextualize the impact findings with rich information 
gathered through the implementation study, which may help explain inter- and intra-state 
variation in the results as well as differences among subgroups. 


Conclusions: The AO evaluation will provide important insights into the feasibility and effects of 
scaling the |-BEST model to additional colleges and states. This program or others similar to it 
may provide a promising approach to address the pressing social issue of promoting education 
and employment success for low-skilled adults. 


Paper 3 Title: College CountS: Evidence of Impact 


Authors: Michael Scuello (mscuello@metisassoc.com) and Robert Harrington 
(rharrington@metisassoc.com) — Metis Associates, Inc. 


Abstract (668 words): 
Background and Setting: 


Arkansas has been at the forefront of national welfare reform efforts in creating a unique 
approach that uses Temporary Assistance to Needy Families (TANF) dollars allocated to target 
education and training to low-income Arkansans. The Career Pathways Initiative (CPI), 
administered by the Arkansas Department of Higher Education (ADHE) and the state’s 22 two- 
year colleges and three technical centers through Arkansas Community Colleges (ACC), has 
provided education and training to more than 30,000 low-income Arkansans since 2006, 
helping them acquire degrees and/or credentials to obtain and hold jobs in selected high- 
demand and high-wage industries. 


Purpose: 


In 2015, a rigorous, phased evaluation of the CPI Initiative, College CountS: Evidence of Impact, 
was launched with support from the Winthrop Rockefeller, Ford, and Annie E. Casey 
Foundations. The primary goal of the first phase of the College CountS research study is to 
determine the potential impacts of CPI participation on income and education outcomes. 
Future phases will include an investigation of possible differential effects on subgroups, use of 
return-on-Investment models to estimate the monetary impacts of CPI from the participant and 
taxpayer points of view, and an investigation into whether participation in CPI has an impact on 
the children/families of participants. 


Population: 


Anonymized administrative data were collected for approximately 30,000 ACC students who 
participated in CPI activities from 2005 through 2013. Over 90% of the population was 
ethnically White (61.1%) or African-American (32.7%), with trace proportions of Hispanic/Latino 
(2.5%) or other ethnicities such as Asian or Native American (1.9%). The Non-CPI ACC student 
population from which comparison students were selected numbered approximately 400,000 
for the same time period. The Non-CPI population was also overwhelmingly ethnically White 
(74.5%) and African-American (20.3%), with smaller traces of Hispanic/Latino (1.7%) and other 
ethnicities (1.3%). The vast majority of CPI participants were female (89.5%), while Non-CPI 
students were relatively more equally distributed across sexes (55.3% female). CPI participants 
averaged 29.6 years of age at program entry (median 28.1 years), while possible comparisons 
were decidedly younger (mean 25.6 years; median 19.9 years). 


Research Design: 


For the first phase of the study, anonymized administrative data from 2005 to 2013 were 
provided from several datasets, including Unemployment Insurance (UI) wage data, TANF data, 


CPI enrollment data, community college enrollment data, and higher education 
completion/graduation data, including attained certificates and degrees. Data were processed 
and separated into seven annual cohorts of CPI participants and comparison pools to which the 
participants could be matched based on several key characteristics including demographics and 
baseline measurements of outcomes. Matching was conducted via estimated propensity scores 
(PSM), with different estimates calculated for each outcome. Matches were selected via the 
greedy-matching algorithm resulting in one-to-one treatment to comparison matches for each 
outcome. Investigated outcomes included wages and higher education certificate (e.g., 
certificates of proficiency, technical certificates) and degree attainment. Outcomes from 
treatment and comparison groups were analyzed using Hierarchical Linear Modeling (HLM), 
multiple linear, or logistic regression techniques as appropriate. 


Findings / Results 


Of particular note, 52.2% of all students who had ever entered into the Career Pathways 
Initiative from 2006 to 2013 had earned at least one higher education academic credential or 
degree. In those same academic years, only 23.7% of the total non-CPI enrolled general 
community college population from across the state had earned at least one higher education 
certificate or degree. Other notable findings to date include: 


e CPI participants consistently (7 of 7 cohorts) earning higher education certificates and 
degrees at a higher rate than matched community college student comparisons. 


e CPI Participants and CPI participants receiving Transitional Employment Assistance (TEA) 
consistently (3 of 3 cohorts) earning higher wages than matched TANF recipient 
comparisons. 


e Matched community college students consistently (6 of 6 cohorts) earning higher wages 
than CPI participants, although CPI participants appear to be closing the gap between 
populations by the last investigated cohort (2011). 


Conclusions 


While promising, further investigation is necessary to fully understand the impacts of the CPI 
program, through use of enhanced datasets and within-group dosage analyses, including the 
planned future phases of the study identified above. 


Paper 4 Title: A Summative Outcomes Evaluation of TAACCCT-Funded Competency-Based 
Education Programs at Three Community Colleges 


Authors: Ann Person (APerson@mathematica-mpr.com) and Jaime Thomas 
(JThomas@mathematica-mpr.com) — Mathematica Policy Research 


Abstract (950 words): 


Between 2011 and 2014, the U.S. Department of Labor (DOL), Employment and Training 
Administration awarded nearly $500 million per year in grants to individual community colleges 
and groups of institutions, through the Trade Adjustment Assistance Community College and 
Career Training (TAACCCT) grants program. The broad goals of the TAACCCT program were (1) 
to increase attainment of postsecondary credentials that provide skills for employment in high- 
wage, high-growth fields; (2) to introduce or replicate innovative and effective job-relevant 
curricula; and (3) to improve employment outcomes for participants, especially economically 
dislocated and low-skilled adult workers. 


In October 2012, DOL awarded a $12 million grant to a three-college consortium led by Sinclair 
Community College (SCC) in Dayton, Ohio, in partnership with Austin Community College (ACC) 
in Austin, Texas and Broward College (BC) in Fort Lauderdale, Florida. Together, the colleges 
developed and implemented competency-based education (CBE) models in information 
technology pathway programs. The CBE approach requires students to master clearly defined 
and measurable learning outcomes (the required “competencies”) but allows variation in the 
time each student takes to demonstrate each competency. The approach contrasts with 
traditional models, in which each student may experience different learning outcomes, but all 
spend a fixed amount of time in each course. 


This paper reports results from the summative outcomes evaluation of the colleges’ TAACCCT- 
funded programs. We use administrative data from student information systems and state 
wage records to answer four research questions: (1) What are the cumulative education and 
employment outcomes of TAACCCT participants? (2) What factors are associated with TAACCCT 
participants’ outcomes? (3) Do outcomes differ for participants exposed to more or less 
TAACCCT-funded programming? (4) How do TAACCCT participants’ outcomes compare to the 
outcomes of similar students in traditional IT programs at the colleges? 


To answer research question (1), we describe the education and employment outcomes of 
students enrolling in grant-funded CBE programs. To answer research question (2), we examine 
correlations between participants’ outcomes and factors such as student sociodemographic 
characteristics. Answering research question (3) involves a dosage analysis to investigate 
whether exposure to more programming is associated with better outcomes. Finally, to answer 
research question (4), we use propensity score matching to compare outcomes of participating 
students and similar, nonparticipating students. We also draw from an earlier implementation 
study of the grant to contextualize the results of the outcomes analyses. 


We summarize findings for each research question below: 
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(1) What are the cumulative education and employment outcomes of TAACCCT participants? 


More than a third (35%) of participants completed a program of study, including industry 
certification preparatory courses, college certificates, or degrees. 


On average, participants completed programs quickly, requiring approximately two terms from 
the time they entered the program to complete their first program of study. For certificates and 
associate’s degrees, participants took approximately four terms after initial enrollment. 


Employment rates started and remained high (over two-thirds), and wages for employed 
participants increased after program enrollment, on average by 13% (compared to national 
wage growth over the same period of about 7%; calculated from Bureau of Labor Statistics 
data). 


(2) What factors are associated with TAACCCT participants’ outcomes? 


Credential completers tended to be older, experienced students; many were enrolled full-time. 
Most participants who completed a certificate or degree were age 25 or older (about two-thirds 
across all three colleges), and a large majority of those who completed credentials had prior 
postsecondary experience (86 percent consortium-wide). About a third (35%) had already 
completed a postsecondary credential and more than half (53%) percent of participants who 
completed credentials enrolled full-time on initial entry into the program. 


CBE programs may help level the playing field for older participants, but other predictors of 
program and credential completion varied. We examined the relationship between a host of 
participant characteristics (age, gender, race/ethnicity, English language status, Pell grant 
eligibility, expected family contribution, high school GPA, developmental course history, prior 
postsecondary experience, prior postsecondary credential, initial enrollment status, incumbent 
worker status, veteran status, and TAA eligibility) and program and credential completion, using 
both bivariate and multivariate approaches. Results consistently showed that age and full-time 
enrollment status were positively and significantly related to program and credential 
completion, but results for other participant characteristics were inconsistent. 


(3) Do outcomes differ for participants exposed to more or less TAACCCT-funded programming? 


Participants with access to fully online, flexibly paced CBE courses and the most enhanced 
academic coaching achieved higher program and credential completion rates. SCC offered 
different combinations of course modality and learner supports to different groups of 
participants, presenting a unique opportunity to examine the relationships between these 
program features and educational outcomes. Multivariate analysis showed the group with 
access to the fully online, flexibly paced CBE courses and the most enhanced academic coaching 
included the highest proportion of program and credential completers. 


(4) How do TAACCCT participants’ outcomes compare to the outcomes of similar students in 
traditional IT programs at the colleges? 
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Gatekeeper course completion rates were slightly lower for participants than for comparison 
students. 


Differences in participants’ and nonparticipants’ credential completion rates varied by college 
and may reflect unobservable differences between the groups. 


Studies of community college pathway programs, like the Sinclair consortium’s TAACCCT 
evaluation, can not only improve our understanding of how to support students’ education and 
career opportunities, but can also make government more effective by informing public 
agencies’ grantmaking and funding strategies. Shedding light on program effectiveness informs 
decisions on the types of programs to fund. Providing information on the factors that influence 
a program’s effectiveness can help funding agencies understand the contexts in which a 
program might be effective and so better target funds. Finally, studies like these can improve 
the quality of funder-mandated evaluations by providing input on the most important factors 
and outcomes to measure in different contexts. 
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